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However, the students did not exhibit any obviously exclusive characteristics. The 
test indicated that the null hypothesis of no statistically significant difference 
between the pretest and posttest scores could be rejected at the p=0/. 00024 level (13 
higher and 0 lower) . It is concluded, then, that there were differences relative to 
the pretest and posttest scores, suggesting that the class was effective for learning 
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Effectiveness of a Graduate Measurement Course 

Abstract 

To determine the effectiveness of a graduate 
educational assessment course in measurement processes, 
this study compared test scores from the beginning and 
from the end of the course. The three sections that 
were the focus of this study were offered in the Fall 
2002, Spring 2003, and Summer 2003 terms with the same 
instructor. The course was offered as a traditional 
face-to-face class using PowerPoint presentations 
during all of the lectures. All sections incorporated 
several quizzes, hands-on activities, and a fifty- 
question multiple-choice final exam. The purpose of the 
activities was to assess the students ' knowledge of the 
basic components involved in educational assessment, 
particularly the measurement process. There were 44 
participants for whom there was complete information, 
comprising 35 females (80%) and 9 males (20%) . 
Multiple-choice pretests and posttests on fundamental 
assessment topics were given. The assumptions for the 
dependent t-test and for the Wilcoxon test could not be 
met, so a quantile (sign) test was run to compare the 
pretest and posttest scores. The assumption that the 
measurement scale be at least ordinal was met since the 
data comprised frequency counts, but random selection 
was not possible since students are not randomly 
assigned to classes. However, the students did not 
exhibit any obviously exclusive characteristics. The 
test indicated that the null hypothesis of no 
statistically significant difference between the 
pretest and posttest scores could be rejected at the 
p=0. 00024 level (13 higher and 0 lower) . It is 
concluded, then, that there were differences relative 
to the pretest and posttest scores, suggesting that the 
class was effective for learning the measurement topics 
introduced. 
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Effectiveness of a Graduate Measurement Course 



The study investigated the effectiveness of a 
graduate educational assessment course in measurement 
processes. The three sections involved in this study 
were offered in the Fall 2002, and Spring and Summer 
2003 terms. There were 44 participants for whom there 
was complete data. All participants were students 
enrolled in the Assessment: Measurement graduate 
course. The design of the study was a single-sample 
pretest-posttest. Frequency counts for correct 
responses were used and the quantile (sign) test 
indicated that the null hypothesis could be rejected at 
the p=0. 00024 level. 

Activities 

The course that is the subject of this study used a 
traditional face-to-face lecture model. PowerPoint 
presentations and collaborative group discussions were 
used to enhance the presentations and to provide a 
student-centered, activity-based course. This course is 
a one-hour course in a three-hour block, or sequence, 
of courses on assessment, comprising 15 hours over a 
five-week period. The activities for the classes 
included a critical analysis of a journal article on 
assessment, 3 quizzes, 2 group activities, and a final 
exam. These group activities were designed to provide 
practice for the students regarding the fundamental 
aspects of assessment and measurement. Participation in 
class discussions was always encouraged. The most 
recent syllabus for the course is appended to this 
paper . 

Design 

The design of the study was a quasi-experimental 
single-sample pretest-posttest. There was no control 
group in this study due to all of the students being 
taught by the same instructor, using the same 
instructional methods and materials. This course is not 
offered by any other instructor at this time. It is 
noted by this researcher that this approach is a weak 
experimental design because of the lack of a control 
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group in this study due to all of the students being 
taught by the same instructor, using the same 
instructional methods and materials. This course is not 
offered by any other instructor at this time. It is 
noted by this researcher that this approach is a weak 
experimental design because of the lack of a control 
group to support the idea that an intervention is the 
reason for any differences between pretest scores and 
posttest scores. In addition, the students who were 
compared in this study were from intact classes, rather 
than randomly assigned groups. Therefore, it may be 
inappropriate to make causal inferences based on the 
results . 

Subjects 

The three sections involved in this study were 
offered in the Fall 2002, and Spring and Summer 2003 
terms. There were 44 participants for whom there was 
complete data. All participants were students enrolled 
in the Assessment: Measurement graduate course. All of 
the students were admitted to various Master' s degree 
programs within the College of Education at the 
university . 

Instrument and Data Analysis 

The pretest instrument was a 15-question multiple 
choice test developed by the instructor. The same 15 
questions were incorporated into the 50-question 
multiple choice final exam which was used as the post- 
test instrument. Frequency counts of the number of 
correct responses were used as scores, and the quantile 
(sign) test indicated that the null hypothesis of no 
difference in the mean scores on the pretests and 
posttests could be rejected at the p=0. 00024 level. 

That is, between the pretest and the posttest, 13 of 
the 15 students had scores that were higher on the 
posttest and 0 had scores that were lower. 

Discussion 
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It is concluded, then, that there were differences 
relative to the pretest and posttest scores, suggesting 
that the class may have been effective for learning the 
measurement topics introduced. However, since a 
limitation of the study is the small sample size, any 
generalizations of these findings would need to be done 
with caution. In addition, it was not possible to have 
a control group since the instructor is the only person 
teaching this particular course. It is possible, then, 
that the change could have arisen from some 
unidentified source other than the instructional 
approach investigated here. While no other cause is 
suspected, that possibility remains. 
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One-Sample T-Test Report 



Page/Date/Time 1 6/29/2003 4:50:36 PM 

Database C:\My Documents\MSERA\PamRobMSERAAbs.SO 

Variable Pretest 

Descriptive Statistics Section 







Standard 


Standard 


95% LCL 


95% UCL 


Variable Count 


Mean 


Deviation 


Error 


of Mean 


of Mean 


Pretest 13 


20.07692 


8.139016 


2.257357 


15.15856 


24.99528 


T for Confidence Limits 


= 2.1788 












Tests of Assumptions Section 












Assumption 




Value 


Probability 


Decision(5%) 






Skewness Normality 




0.8643 


0.387433 


Cannot reject normality 




Kurtosis Normality 




0.1974 


0.843480 


Cannot reject normality 




Omnibus Normality 




0.7860 


0.675040 


Cannot reject normality 




Correlation Coefficient 














T-Test For Difference Between Mean and Value Section 








Alternative 






Prob 


Decision 


Power 


Power 


Hypothesis 




T-Value 


Level 


(5%) 


(Alpha=.05) 


(Alpha=.01) 


Pretest<>0 




8.8940 


0.000001 


Reject Ho 


1.000000 


1.000000 


Pretest<0 




8.8940 


0.999999 


Accept Ho 


0.000000 


0.000000 


Pretest>0 




8.8940 


0.000001 


Reject Ho 


1.000000 


1.000000 


Nonparametric Tests Section 












Quantile (Sign) Test 














Hypothesized 




Number 


Number 


Prob 


Prob 


Prob 


Value 


Quantile 


Lower 


Higher 


Lower 


Higher 


Both 


0 


0.5 


0 


13 


0.000122 


1 .000000 


0.000244 



Wilcoxon Signed-Rank Test for Difference in Medians 



W 

Sum Ranks 
91 



Mean 
of W 

45.5 



Std Dev Number 

ofW of Zeros 

14.30035 0 



Number Sets Multiplicity 
of Ties Factor 

2 12 



Alternative 

Hypothesis 

Median<>0 

Median<0 

Median>0 



Exact Probability 
Prob Decision 
Level (5%) 



Approximation Without 
Continuity Correction 

Prob Decision 
Z-Value Level (5%) 

3.1817 0.001464 Reject Ho 

3.1817 0.999268 Accept Ho 

3.1817 0.000732 Reject Ho 



Approximation With 
Continuity Correction 

Prob Decision 
Z-Value Level (5%) 

3.1468 0.001651 Reject Ho 

3.2167 0.999352 Accept Ho 

3.1468 0.000825 Reject Ho 
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One-Sample T-Test Report 



Page/Date/Time 

Database 

Variable 

Plots Section 



2 6/29/2003 4:50:36 PM 

C:\My Documents\MSERA\PamRobMSERAAbs.SO 
Pretest 



Histogram of Pretest 




Average-Difference Plot 
23.0-. o 



Normal Probability Plot of Pretest 




19.0-1— r 
5.0 



-i — i — i — i — i — i — i — i—O — i — i — i — i — i — i — i — i 

11.3 17.5 23.8 30.0 

Difference 
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One-Sample T-Test Report 



Page/Date/Time 3 6/29/2003 4:50:36 PM 

Database C:\My Documents\MSERA\PamRobMSERAAbs.SO 

Variable Postest 

Descriptive Statistics Section 







Standard 


Standard 


95% LCL 


95% UCL 


Variable Count 


Mean 


Deviation 


Error 


of Mean 


of Mean 


Postest 1 3 


34.38462 


7.006407 


1 .943228 


30.15069 


38.61855 


T for Confidence Limits 


= 2.1788 












Tests of Assumptions Section 












Assumption 




Value 


Probability 


Decision(5%) 






Skewness Normality 




-1.5935 


0.111047 


Cannot reject normality 




Kurtosis Normality 




0.1764 


0.859945 


Cannot reject normality 




Omnibus Normality 




2.5704 


0.276596 


Cannot reject normality 




Correlation Coefficient 














T-Test For Difference Between Mean and Value Section 








Alternative 






Prob 


Decision 


Power 


Power 


Hypothesis 




T-Value 


Level 


(5%) 


(Alpha=.05) 


(Alpha=.01) 


PostestoO 




17.6946 


0.000000 


Reject Ho 


1.000000 


1.000000 


Postest<0 




17.6946 


1 .000000 


Accept Ho 


0.000000 


0.000000 


Postest>0 




17.6946 


0.000000 


Reject Ho 


1.000000 


1.000000 


Nonparametric Tests Section 












Quantile (Sign) Test 














Hypothesized 




Number 


Number 


Prob 


Prob 


Prob 


Value 


Quantile 


Lower 


Higher 


Lower 


Higher 


Both 


0 


0.5 


0 


13 


0.000122 


1,000000 


0.000244 



Wilcoxon Signed-Rank Test for Difference in Medians 



W 




Mean 


Std Dev Number 


Number Sets Multiplicity 


Sum Ranks 




of W 


ofW of Zeros 


of Ties 


Factor 




91 




45.5 


14.30472 0 




1 


6 










Approximation Without 


Approximation With 






Exact Probability 


Continuity Correction 


Continuity Correction 




Alternative 


Prob 


Decision 


Prob 


Decision 




Prob 


Decision 


Hypothesis 


Level 


(5%) 


Z-Value Level 


(5%) 


Z-Value 


Level 


(5%) 


Median<>0 






3.1808 0.001469 


Reject Ho 


3.1458 


0.001656 


Reject Ho 


Median<0 






3.1808 0.999266 


Accept Ho 


3.2157 


0.999349 


Accept Ho 


Median>0 






3.1808 0.000734 


Reject Ho 


3.1458 


0.000828 


Reject Ho 
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One-Sample T-Test Report 



Page/Date/Time 

Database 

Variable 

Plots Section 



4 6/29/2003 4:50:36 PM 

C:\My Documents\MSERA\PamRobMSERAAbs.SO 
Postest 



Histogram of Postest 



Normal Probability Plot of Postest 




Average-Difference Plot 

37 . 0 -. 

" o 

o o 

35.5- 




<D 

O) 

CO 



34.0- 



O 



32.5- 



3.1. 0-1— r 
0.0 



1 1 r 

6.3 



i t 1 r — i 



12.5 

Difference 



o 

o 

T 1 1 I 1 1 1 

18.8 25.0 
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University of Arkansas at Little Rock 
College of Education 
Course Outline/Syllabus 
Department of Educational Leadership 



I. Course Prefix and Number 

II. Course Title 

III. Credit 

IV. Semester and Year 

V. Instructor: 

VI. Office Location 

VII. Office Hours 

VUL. Telephone and E-mail: 



EDFN7171 

Educational Assessment: Measurement Process 
1 hour 

Summer 2003 
Pamela Broadston, M.Ed. 

419J Dickinson 
By appointment 
xxx-xxxx 
xxx @ualr.edu 



IX. Course Description 

This course is part of a three-course sequence (EDFN 7171, 7172, 7173) treating advanced topics in assessment and 
evaluation. The focus is on the role of measurement in education and human service agencies. It also deals with the 
psychometric properties of norm-referenced and criterion-referenced tests. 

The Conceptual Framework for programs in the College of Education is Leadership in Learning through the Specialized 
Expertise (SE), Communication (C), and Professional Development (PD). 

This course is designed to prepare teachers to meet the Principles of Licensure adopted by the Arkansas Department of 
Education. 

Principle #1 : The teacher understands the central concepts, tools of inquiry, and structures of the discipline 
(s) he or she teachers, can create learning experiences that make these aspects of subject matter meaningful 
for students can link the discipline(s) to other subjects. 

Principle #2: The teacher plans curriculum appropriate to the students, to the content, and to the course 
objectives. 

Principle #3: The teacher plans instruction based upon human growth and development, learning theory, 
and the needs of students. 

Principle #4: The teacher exhibits human relations skills which support the development of human potential. 

Principle #5: The teacher works collaboratively with school colleagues, parents/guardians, and the 
community to support students’ learning and well being. 

XI. Course Objectives 

1 . Students will discuss how the assessment process may be used to verify and specify student performance problems, 
identify the kinds of assessment data that are necessary for particular decisions in educational settings, and evaluate 
how perspectives toward special populations can influence the interpretation and use of assessment data. (SE, PI; 
1.2.2, 1.2.3, 1.2.4) 

2. When presented with test scores for hypothetical children, students will interpret these children’s performances when 
they are expressed as grade/age equivalents, percentile ranks, and standard scores, taking into account the impact that 
the standard error of measurement has on these derived scores. (SE, C, PI, P4; Communicator 3.2.1) 

3. When given demographic data about a particular child-such as socioeconomic status, ethnic origin, handicapping 
condition, and geographic location-students will review sections in test manuals on standardization and determine 
whether the child should be evaluated using these tests. (SE, PD, PI; Instructor 1.2.2, 1.2.3, 1.2.4) 

- 10 - 





4. Ater completing readings and participating in class discussions regarding technical adequacy of tests, student will 

evaluate the technical adequacy of assessment information on interpretations and decisions pertaining to groups and 
individuals. (SE, PD, PI; Instructor 1.2.3, Communicator 3.2.1) 

XI. Methods/Instructional Strategies, including Text Readings and Instructional Resources 
The primary methods for this course will be lecture and group discussion. 

Text: 

Kubiszyn, T., & Borich, G. (2003). Educational testing and measurement : Classroom application and practice. (7 th ed.). 
New York: John Wiley & Sons, Inc. 

XU. Assignments, Evaluation Procedures, and Grading Polices 

Assignments 

All class members will be expected to come to class having read the assignment so they can participate intelligently in class 
discussions. Students will be required to take one exam, complete three quizzes/activities, complete one critical analysis and 
participate in classroom discussions. 

Evaluation Procedures 

The following grading model will be used: 



1 exam at 50 points 50 

3 quizzes/activities at 15 points each 45 

1 critical analysis 25 points 25 

Total points possible 120 



Grading Policy 

Grades will be assigned on the following basis: 



90% - 100% A 

80% - 89% B 

70% - 79% . C 

69% and below F 



XIII. Class Policies 
Attendance 

Students will be expected to attend all scheduled sessions of the course. Missing one session is often equivalent to missing an 
entire unit of instruction. The course is very cumulative, and losing an entire unit would almost certainly result in a serious 
loss of continuity for the student. Documented illness, family emergency, official conflicts, and the like will be treated as 
excused absences, but it is ultimately the student’s responsibility to maintain his/her standing in the course. The student 
should make every attempt to contact the instructor prior to being absent. Make-up exams, quizzes, and projects will be 
given only if accompanied by a valid DOCUMENTED excuse . Projects will be accepted after the due date only if 
accompanied by a valid DOCUMENTED excuse . The student is responsible for contacting the instructor in the event of a 
missed exam, quiz/class activity, and/or project prior to the next attended class session. 

Class Participation : Each student is expected to read assigned material prior to class, to participate in class discussions, and to 
complete assignments in a timely manner. 

Cell Phones/Pagers : Please turn off during class. If you are concerned that emergency calls may occur, please discuss with 
instructor. 

Academic Dishonesty : Please refer to the UALR Student Handbook for definitions of academic dishonesty. The handbook is 
available online: http://www.ualr. edu/handbook/STUD ENT HANDBOOK 2002 2003.HTM 

Any instance of academic dishonesty will result in the student receiving a grade of F for the exam or assignment. The offense 
will be reported to the Dean of Students as per UALR policy. 

- 11 - 
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XIV. Class Schedule/Topical Outline - Items in the Schedule are subject to change 



Dates 
May 27 



Topics 



Chapters 



Review of Syllabus 

Introduction to Testing and Measurement 
High Stakes Testing 



1,3 

2 



May 29 



Quiz 

Norm-referenced & Criterion-referenced Tests 
Basic Statistics 



4 

12 



June 3 



Quiz / activity 

Variability, Normal Distribution, & Converted Scores 

Correlation 

Critical Analysis Due 



13 

14 



June 6 



Quiz / activity 
Validity & Reliability 
Accuracy and Error 



15, 16 
17 



June 10 



Exam (50 multiple choice questions) 
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accommodations. Students with disabilities also are encouraged to contact Disability Support Services, which is located in the 
Donaghey Student Center, Room 103, telephone 569-3143, and on the Web at http:www.ualr.edu/~dssdept/index.html . 
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